GENERATIVE MODEL

TruthReader: Towards Trustworthy Document Assistant Chatbot with Reliable Attribution (2024.emnlp-demo)

Challenge:	Recent prompt learning has received significant attention, where downstream tasks are reformulated to the mask-filling task with the help of a textual prompt.
Approach:	They propose a model PromptGen which can automatically generate prompts conditional on the input sentence.
Outcome:	The proposed model outperforms baseline models on the knowledge probing LAMA benchmark.

Copied to clipboard

Dongfang Li, Xinshuo Hu, Zetian Sun, Baotian Hu, Shaolin Ye, Zifei Shan, Qian Chen, Min Zhang

Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech (2021.findings-acl)

Challenge:	Document assistant chatbots are empowered with extensive capabilities by Large Language Models (LLMs) however, they suffer from hallucinations that are difficult to verify in the context of given documents.
Approach:	They propose a document assistant chatbot with reliable attribution that enables users to seek relevant information from given documents.
Outcome:	The proposed system generates answers with detailed inline citations, which can be attributed to the original document paragraphs, facilitating verification of factual consistency of the generated text.

Copied to clipboard

Wanzheng Zhu, Suma Bhat

Ethical NLP

Challenge:	Off-the-shelf methods to generate hate speech are limited in that they generate repetitive and safe responses regardless of the hate speech.
Approach:	They propose a three-module pipeline approach to generate diverse and relevant counterspeech . they first generate various counterspeak candidates by a generative model, then filter ungrammatical ones using a BERT model .
Outcome:	The proposed pipeline generates diverse and relevant counterspeech responses on three datasets.

A Generative Approach for Mitigating Structural Biases in Natural Language Inference (2022.starsem-1)

Copied to clipboard

Dimion Asael, Zachary Ziegler, Yonatan Belinkov

WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning (2023.acl-long)

Challenge:	Natural language inference datasets contain artifacts and biases that allow models to perform poorly by using a biased subset of the input without considering the remainder features.
Approach:	They reformulate a natural language inference task as a generative task . they find that this approach is highly robust to large amounts of bias .
Outcome:	The proposed model is highly robust to large amounts of bias.

Copied to clipboard

Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Sujian Li, Yajuan Lyu

Align and Augment: Generative Data Augmentation for Compositional Generalization (2024.eacl-long)

Challenge:	Existing factual consistency metrics are often uncontrollably generating text that is factually inconsistent with inputs.
Approach:	They propose a weakly supervised framework that is directly trained on actual generated samples from language models with weakly annotated labels.
Outcome:	The proposed framework improves on the TRUE benchmark by 3.3% over existing methods with 435M parameters.

Copied to clipboard

Francesco Cazzaro, Davide Locatelli, Ariadna Quattoni

Modeling Sense Structure in Word Usage Graphs with the Weighted Stochastic Block Model (2021.starsem-1)

Challenge:	Recent work on semantic parsing has shown that seq2seq models find compositional generalization challenging.
Approach:	They propose a data-augmentation strategy that exploits alignment annotations between sentences and their corresponding meaning representations to improve compositional generalization.
Outcome:	The proposed model improves compositional generalization performance by exploiting alignment annotations between sentences and their corresponding meaning representations.

Copied to clipboard

Dominik Schlechtweg, Enrique Castaneda, Jonas Kuhn, Sabine Schulte im Walde

Structured Data in NLP

Challenge:	Word Usage Graphs capture fine-grained semantic proximity distinctions between word uses.
Approach:	They propose to model word use Graphs using a Bayesian weighted stochastic block model and a probabilistic weightes-based model to capture fine-grained semantic proximity distinctions between word uses.
Outcome:	The proposed model is compared with existing models and is empirically most adequate.

Tackling Long-Tailed Relations and Uncommon Entities in Knowledge Graph Completion (D19-1)

Copied to clipboard

Zihao Wang, Kwunping Lai, Piji Li, Lidong Bing, Wai Lam

Event Causality Identification via Generation of Important Context Words (2022.starsem-1)

Challenge:	Recent studies have focused on the large proportion of infrequent relations which have been ignored by previous studies.
Approach:	They propose a meta-learning framework that aims at handling infrequent relations with few-shot learning and uncommon entities by using textual descriptions.
Outcome:	The proposed framework outperforms existing methods when dealing with infrequent relations and uncommon entities.

Copied to clipboard

Hieu Man, Minh Nguyen, Thien Nguyen

Challenge:	Prior work focused on identifying causal relation between two event mentions . current models do not output important contexts for causal prediction of two mentions.
Approach:	They propose to use dependency path generation as a complementary task for ECI.
Outcome:	The proposed model can generate both causal relation and dependency path words from input sentences.

Large Scale Generative Multimodal Attribute Extraction for E-commerce Attributes (2023.acl-industry)

Copied to clipboard

Anant Khandelwal, Happy Mittal, Shreyas Kulkarni, Deepak Gupta

Consistent Text Categorization using Data Augmentation in e-Commerce (2023.acl-industry)

Challenge:	E-commerce websites often don’t label or mislabel attributes of products .
Approach:	They propose a multi-modal product attribute generation system that extracts product attributes from the product pages of eCommerce stores by using both text and images.
Outcome:	The proposed model improves the recall@90P accuracy by 10.16% and 6.9 from the state-of-the-art models.

Copied to clipboard

Noa Avigdor, Guy Horowitz, Ariel Raviv, Stav Yanovsky Daye

A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings (2020.tacl-1)

Challenge:	Upon closer inspection, we found inconsistencies in the labeling of similar items.
Approach:	They propose to improve an existing product categorization model that takes a product title as input and outputs the most suitable category out of thousands of available candidates.
Outcome:	The proposed model is based on a product title and outputs the most suitable category out of thousands of available candidates.

Copied to clipboard

Lixing Zhu, Yulan He, Deyu Zhou

Semantic Parsing with Syntax- and Table-Aware SQL Generation (P18-1)

Challenge:	Experimental results show that the proposed model outperforms word-level embedding methods in word similarity evaluation and word sense disambiguation.
Approach:	They propose a generative model that explores local and global context for joint learning topics and topic-specific word embeddings.
Outcome:	The proposed model outperforms word-level embedding methods in word similarity evaluation and word sense disambiguation.

Copied to clipboard

Yibo Sun, Duyu Tang, Nan Duan, Jianshu Ji, Guihong Cao, Xiaocheng Feng, Bing Qin, Ting Liu, Ming Zhou

skweak: Weak Supervision Made Easy for NLP (2021.acl-demo)

Challenge:	Existing approaches generate a SQL query word-by-word but results are incorrect or not executable due to mismatch between question words and table contents.
Approach:	They propose a generative model to map natural language questions into SQL queries.
Outcome:	The proposed model significantly improves state-of-the-art execution accuracy from 69.0% to 74.4% on a large question- SQL dataset.

Copied to clipboard

Pierre Lison, Jeremy Barnes, Aliaksandr Hubin

Low-Resource NLP

Challenge:	skweak is a Python-based toolkit for NLP developers to use weak supervision . labelled data remains a scarce resource in many practical NLP scenarios .
Approach:	They present a Python-based toolkit that allows NLP developers to use weak supervision . skweak is designed to facilitate the use of weak supervision for NLP tasks .
Outcome:	skweak is a Python-based toolkit that facilitates weak supervision . the toolkit provides a simple interface to apply labels to a large corpus of text data .

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer (2021.naacl-main)

Copied to clipboard

Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel

Text Generation

Challenge:	Current natural language processing pipelines often use transfer learning, where a model is pre-trained on a data-rich task before being fine-tuned on . this significantly limits their use given that roughly 80% of the world population does not speak English.
Approach:	They introduce a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages.
Outcome:	The proposed model achieves state-of-the-art on multilingual benchmarks and a simple technique to prevent accidental translation in the zero-shot setting.

Aligning Vector-spaces with Noisy Supervised Lexicon (N19-1)

Copied to clipboard

Noa Yehezkel Lubin, Jacob Goldberger, Yoav Goldberg

Representation Learning

Challenge:	Current approaches to learning to translate between two vector spaces assume that the lexicon defines alignment pairs is noise-free.
Approach:	They propose a model that accounts for noisy pairs and propose supervised learning problems for this problem.
Outcome:	The proposed model significantly improves translation accuracy on bilingual word embedding translation and mapping between diachronic embeddable spaces.

A Generative Model for Lambek Categorial Sequents (2024.lrec-main)

Copied to clipboard

Jinman Zhao, Gerald Penn

Challenge:	generative models such as PLC+ generate grammatical sentences with a high probability of being grammatized.
Approach:	They propose a generative model, PLC+, for generating Lambek Categorial Grammar(LCG) sequents.
Outcome:	The proposed model generates Lambek Categorial Grammar(LCG) sequents and is more robust to probabilistic context-free grammars.

CoUDA: Coherence Evaluation via Unified Data Augmentation (2024.naacl-long)

Copied to clipboard

Dawei Zhu, Wenhao Wu, Yifan Song, Fangwei Zhu, Ziqiang Cao, Sujian Li

De-Bias for Generative Extraction in Unified NER Task (2022.acl-long)

Challenge:	Existing data augmentations for coherence evaluation rely on heuristic rules and lack designing criteria.
Approach:	They propose a data augmentation framework that breaks down coherence into global and local aspects and designs augmentation strategies for both aspects.
Outcome:	The proposed framework surpasses recent models in scoring and ranking tasks with 233M parameters.

Copied to clipboard

Shuai Zhang, Yongliang Shen, Zeqi Tan, Yiquan Wu, Weiming Lu

Learning from LLM Agents: In-Context Generative Models for Text Casing in E-Commerce Ads (2025.emnlp-industry)

Challenge:	Existing methods for Named entity recognition (NER) are not consistent with the task, which makes the model vulnerable to incorrect biases.
Approach:	They propose to use generative model to recognize entities from sentences . they analyze incorrect biases in the generation process from a causal perspective .
Outcome:	The proposed method improves the performance of the generative NER model in various datasets.

Copied to clipboard

Yingxue Zhou, Tan Zhu, Tao Zeng, Zigeng Wang, Wei Shen

Optimizing Retrieval-augmented Reader Models via Token Elimination (2023.emnlp-main)

Challenge:	Existing NER-based transformer models are expensive and lack contextual dependencies, making them less reliable when handling unseen or ad-specific terms, e.g., brand names.
Approach:	They propose a two-stage approach to casing correction in e-commerce ad content that leverages Chain-of-Actions to enforce content policies while accurately handling ads-specific terms.
Outcome:	The proposed model outperforms existing NER-based models and achieves near-LLM performance at a fraction of the cost.

Copied to clipboard

Moshe Berchansky, Peter Izsak, Avi Caciularu, Ido Dagan, Moshe Wasserblat

COSIGN: Contextual Facts Guided Generation for Knowledge Graph Completion (2024.naacl-long)

Challenge:	Existing methods for ODQA use a retrieval-augmented language model . a generative model can cause a significant bottleneck in decoding time .
Approach:	They propose to eliminate some of the retrieved information that might not contribute essential information to the answer generation process.
Outcome:	The proposed method reduces run-time by up to 62.2% with only 2% reduction in performance and improves performance.

Copied to clipboard

Jinpeng Li, Hang Yu, Xiangfeng Luo, Qian Liu

Structured Data in NLP

Challenge:	Existing methods for knowledge graph completion (KGC) are limited in generality and scalability due to poor contextual facts.
Approach:	They propose a contextual facts collector and contextual facts organizer to enhance the inference ability of GM-based methods for various KGC tasks.
Outcome:	The proposed model outperforms state-of-the-art methods in terms of performance.

Generative Text Modeling through Short Run Inference (2021.eacl-main)

Copied to clipboard

Bo Pang, Erik Nijkamp, Tian Han, Ying Nian Wu

Challenge:	Latent variable models for text capture global semantic and syntactic features when trained correctly.
Approach:	They propose a short run dynamics for inference that initializes from the prior distribution of the latent variable and runs a small number of Langevin dynamics steps guided by its posterior distribution.
Outcome:	The proposed model is able to generate coherent sentences with smooth transition and shows no sign of posterior collapse.

BLM-AgrF: A New French Benchmark to Investigate Generalization of Agreement in Neural Networks (2023.eacl-main)

Copied to clipboard

Aixiu An, Chunyang Jiang, Maria A. Rodriguez, Vivi Nastase, Paola Merlo

Challenge:	Existing benchmarks for deep learning are based on massive amounts of data, which are effective in hiding some of the shallowness of the learned models.
Approach:	They propose to use a French dataset to learn the underlying rules of subject-verb agreement in sentences, inspired by visual IQ tests known as Raven’s Progressive Matrices.
Outcome:	The proposed method is based on Raven’s Progressive Matrices, a visual IQ test, and a dataset built using the BLM framework.

MixQG: Neural Question Generation with Mixed Answer Types (2022.findings-naacl)

Copied to clipboard

Lidiya Murakhovs’ka, Chien-Sheng Wu, Philippe Laban, Tong Niu, Wenhao Liu, Caiming Xiong

Question Generation

Challenge:	Existing neural question generation approaches focus on short factoid type of answers.
Approach:	They propose a neural question generator that trains a single generative model by combining multiple question types with different answer types.
Outcome:	The proposed model outperforms existing models in both seen and unseen domains and can generate questions with different cognitive levels when conditioned on different answer types.

Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021.acl-short)

Copied to clipboard

Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang

Adversarial DPO: Harnessing Harmful Data for Reducing Toxicity with Minimal Impact on Coherence and Evasiveness in Dialogue Agents (2024.findings-naacl)

Challenge:	Recent work focuses on question answering based on machine reading comprehension . current approaches treat QA as extracting a consecutive piece of text to a given question.
Approach:	They propose a generative QA model that incorporates an extractive mechanism into a model.
Outcome:	The proposed model improves quality and semantic accuracy over baseline models.

Copied to clipboard

San Kim, Gary Lee

Learning to Generate Questions by Learning to Recover Answer-containing Sentences (2021.findings-acl)

Challenge:	Existing toxicity within large language models can negatively impact the user experience, causing performance degradation.
Approach:	They propose an adversarial DPO algorithm that improves direct preference optimization (DPO) by incorporating harmful data into the generative model.
Outcome:	The proposed training algorithm improves the model’s resilience against harmful conversations while minimizing performance degradation.

Copied to clipboard

Seohyun Back, Akhil Kedia, Sai Chetan Chinthakindi, Haejun Lee, Jaegul Choo

SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising (2022.findings-naacl)

Challenge:	Recent research has focused on synthetically generating a question from a given context and an annotated answer by training an additional generative model.
Approach:	They propose a method that learns to generate contextually rich questions by recovering answer-containing sentences.
Outcome:	The proposed approach improves the quality and accuracy of existing models and achieves comparable results to the state-of-the-art on MS MARCO and NewsQA.

Copied to clipboard

Kuan Xu, Yongbo Wang, Yongliang Wang, Zihao Wang, Zujie Wen, Yang Dong

Programming Languages in NLP

Challenge:	Using sketch-based slot filling, text-to-SQL models suffer from over-complexity . et al., e.al., and d.albert, dr., propose a novel method for text- to-Sql generation .
Approach:	They propose to train sequence-to-sequence model with Schema-aware Denoising . they propose a clause-sensitive execution guided (EG) decoding strategy .
Outcome:	The proposed method improves performance in schema linking and grammar correctness . it also establishes new state-of-the-art on the WikiSQL benchmark .

Bi-Phone: Modeling Inter Language Phonetic Influences in Text (2023.acl-long)

Copied to clipboard

Abhirut Gupta, Ananya B. Sai, Richard Sproat, Yuri Vasilevski, James Ren, Ambarish Jash, Sukhdeep Sodhi, Aravindan Raghuveer

Syntactic Text Processing

Challenge:	Increasingly, people are forced to use the Web in languages they have low literacy in due to technology asymmetries.
Approach:	They propose a method to mine phoneme confusions for pairs of L1 and L2 and plug them into a generative model for synthetically producing corrupted L2 text.
Outcome:	The proposed method corrupts the popular language understanding benchmark SuperGLUE and improves performance.

Retrieving Multimodal Prompts for Generative Visual Question Answering (2023.findings-acl)

Copied to clipboard

Timothy Ossowski, Junjie Hu

Unsupervised Learning of Syntactic Structure with Invertible Neural Projections (D18-1)

Challenge:	Visual question answering (VQA) is a multimodal machine learning problem that challenges a model to answer a question posed about an image.
Approach:	They propose a generative model enhanced by multimodal prompt retrieval that integrates retrieved prompts and multimodal features to generate answers in free text.
Outcome:	The proposed model outperforms its non-retrieval counterpart by 30% on medical VQA tasks.

Copied to clipboard

Junxian He, Graham Neubig, Taylor Berg-Kirkpatrick

A Generative Model for Joint Natural Language Understanding and Generation (2020.acl-main)

Challenge:	Unsupervised learning of syntactic structure is typically performed using generative models with discrete latent variables and multinomial parameters.
Approach:	They propose a generative model that jointly learns discrete syntactic structure and continuous word representations in an unsupervised fashion by cascading an invertible neural network with a structured generative prior.
Outcome:	The proposed model outperforms state-of-the-art models on part-of speech (POS) induction and unsupervised dependency parsing without gold POS annotation.

Copied to clipboard

Bo-Hsiang Tseng, Jianpeng Cheng, Yimai Fang, David Vandyke

Dynamic Prefix as Instructor for Incremental Named Entity Recognition: A Unified Seq2Seq Generation Framework (2025.findings-acl)

Challenge:	Natural language understanding (NLU) and natural language generation (NLG) have opposite goals.
Approach:	They propose a generative model which couples NLU and NLG through a shared latent variable.
Outcome:	The proposed model achieves state-of-the-art performance on two dialogue datasets with flat and tree-structured formal representations.

Copied to clipboard

Zihao Wu, YongXiang Hua, Yongxin Zhu, Fang Zhang, Linli Xu

GenerativeRE: Incorporating a Novel Copy Mechanism and Pretrained Model for Joint Entity and Relation Extraction (2021.findings-emnlp)

Challenge:	Named Entity Recognition (NER) is a fundamental problem in information extraction.
Approach:	They propose a parameter-efficient method for Incremental Named Entity Recognition (INER) task aimed at updating a model to extract entities from an expanding set of entity type candidates by employing a dynamic prefix as a task instructor to guide the generative model.
Outcome:	Empirical results show that the proposed method preserves task-invariant knowledge while adapting to new entities with minimal parameter updates.

Copied to clipboard

Jiarun Cao, Sophia Ananiadou

SharpSeq: Empowering Continual Event Detection through Sharpness-Aware Sequential-task Learning (2024.naacl-long)

Challenge:	Existing models for extracting relation triplets suffer from incompletion and disorder problems when they extract multi-token entities from input sentences.
Approach:	They propose a special entity labelling method that fine-tunes the pre-trained model and learns the special entity labels simultaneously.
Outcome:	The proposed model achieves 4.6% and 0.9% improvement over current methods in the NYT24 and NYT29 benchmark datasets.

Copied to clipboard

Thanh-Thien Le, Viet Dao, Linh Nguyen, Thi-Nhung Nguyen, Linh Ngo, Thien Nguyen

Learning to Synthesize Data for Semantic Parsing (2021.naacl-main)

Challenge:	Existing methods for continual event detection suffer from catastrophic forgetting . a novel continual learning paradigm leveraging sharpness-aware minimization is needed .
Approach:	They propose a continual learning paradigm that leverages sharpness-aware minimization and a generative model to balance training data distribution.
Outcome:	The proposed approach outperforms existing methods on real-world datasets.

Copied to clipboard

Bailin Wang, Wenpeng Yin, Xi Victoria Lin, Caiming Xiong

A Deep Factorization of Style and Structure in Fonts (D19-1)

Challenge:	Existing methods for synthesizing data for semantic parsing require handcrafted rules to synthesize new programs or utterance-program pairs.
Approach:	They propose to use a (non-neural) PCFG to model the composition of programs and a BART-based translation model to map a program to an utterance to learn a generative model from existing data.
Outcome:	The proposed model can be efficiently learned from existing data on benchmarks of GeoQuery and Spider.

Copied to clipboard

Nikita Srivatsan, Jonathan Barron, Dan Klein, Taylor Berg-Kirkpatrick

Challenge:	Using a variational inference procedure, we factor each training glyph into a combination of a character-specific content embedding and a latent font-specific style variable.
Approach:	They propose a deep factorization model that disentangles content from style by factorizing each training glyph into a latent content embedding and a learned embeddable character.
Outcome:	The proposed model outperforms a strong nearest neighbors baseline and state-of-the-art discriminative model on reconstructing missing glyphs from an unknown font given only a small number of observations.

Doc2hash: Learning Discrete Latent variables for Documents Retrieval (N19-1)

Copied to clipboard

Yifei Zhang, Hao Zhu

Improving Chinese Grammatical Error Detection via Data augmentation by Conditional Error Generation (2022.findings-acl)

Challenge:	Learning to hash via generative model is a powerful paradigm for fast similarity search in documents retrieval.
Approach:	They propose a method that trains a generative model to generate hash codes by using continuous relaxation on priors.
Outcome:	The proposed method outperforms other state-of-the-art methods in qualitative and quantitative experiments.

Copied to clipboard

Tianchi Yue, Shulin Liu, Huihui Cai, Tao Yang, Shengkang Song, TingHao Yu

Low-Resource NLP

Challenge:	Chinese Grammatical Error Detection is a non-automatic method to detect grammatical errors in texts.
Approach:	They propose a Conditional Non-Autoregressive Error Generation model for Chinese grammatical errors that uses a masking and prediction method to generate a context-dependent error.
Outcome:	The proposed method achieves better performance than all compared data augmentation methods on the CGED-2018 and CGAD-2020 benchmarks.

Neural Generative Rhetorical Structure Parsing (D19-1)

Copied to clipboard

Amandla Mabona, Laura Rimell, Stephen Clark, Andreas Vlachos

Weak Supervision for Learning Discourse Structure (D19-1)

Challenge:	Existing approaches to RST parsing use discriminative models, but these are less sample efficient and datasets are typically small.
Approach:	They propose a document-level RNN grammar with a bottom-up traversal order and a beam search algorithm that keeps track of both structure-and word-generating actions without exhibit-ing this branching bias.
Outcome:	The proposed model outperforms a discriminative model with the same features by 2.6 F1points and achieves comparable performance to the state-of-the-art.

Copied to clipboard

Sonia Badene, Kate Thompson, Jean-Pierre Lorré, Nicholas Asher

Exploring Graph Pre-training for Aspect-based Sentiment Analysis (2023.findings-emnlp)

Challenge:	a weak supervision approach is a promising tool for learning discourse structure for multi-party dialogue.
Approach:	They propose a data programming paradigm that allows a user to label training data using expert-composed heuristics and transform them into probability distributions of the class labels.
Outcome:	The proposed approach outperforms both deep learning and traditional ML approaches on the task of learning discourse structure for multi-party dialogue.

Copied to clipboard

Xiaoyi Bao, Zhongqing Wang, Guodong Zhou

Bot-Adversarial Dialogue for Safe Conversational Agents (2021.naacl-main)

Challenge:	Existing studies tend to extract the sentiment elements in a generative manner to avoid complex modeling of sentiment elements.
Approach:	They propose a generative model with an Element-level Graph Pre-training paradigm and a Task Decomposition Pre- training paradigm to make it generalizable and robust against irregular sentiment quadruples.
Outcome:	The proposed model is generalizable and robust against irregular sentiment quadruples.

Copied to clipboard

Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan

Robustness in NLP

Challenge:	a new method for evaluating chatbot safety is proposed to mimic human-generated data . a bot-adversarial dialogue model learns undesirable features from this data, a study finds .
Approach:	They propose a human-and-model-in-the-loop framework for evaluating toxicity of chatbots . they propose two methods for safe conversational agents by either training on data or ”baking-in” safety to the generative model itself.
Outcome:	The proposed methods are safer than existing models while maintaining usability metrics, the authors say . they show that the proposed methods can be used to make safer models with human-model interactions .

A Batch Normalized Inference Network Keeps the KL Vanishing Away (2020.acl-main)

Copied to clipboard

Qile Zhu, Wei Bi, Xiaojiang Liu, Xiyao Ma, Xiaolin Li, Dapeng Wu

Challenge:	Variational Autoencoder (VAE) is widely used to approximate a model’s posterior on latent variables.
Approach:	They propose to let the Kullback–Leibler divergence individual follow a distribution across the whole dataset and analyze that it is sufficient to prevent posterior collapse by keeping the expectation of the KL’s distribution positive.
Outcome:	The proposed approach can avoid posterior collapse effectively and efficiently without introducing any new model component or modifying the objective.

PseudoReasoner: Leveraging Pseudo Labels for Commonsense Knowledge Base Population (2022.findings-emnlp)

Copied to clipboard

Tianqing Fang, Quyet V. Do, Hongming Zhang, Yangqiu Song, Ginny Y. Wong, Simon See

A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations (N19-1)

Challenge:	Commonsense Knowledge Base (CSKB) Population aims at reasoning over unseen entities and assertions on CSKBs, but it requires out-of-domain generalization ability as the source CSMB for training is of a relatively smaller scale (1M) .
Approach:	They propose a semi-supervised learning framework that uses a teacher model to provide pseudo labels on the unlabeled candidate dataset for a student model to learn from.
Outcome:	The proposed framework can improve the backbone model KG-BERT (RoBERTa-large) by 3.3 points on the overall performance and especially, 5.3 points on out-of-domain performance.

Copied to clipboard

Mingda Chen, Qingming Tang, Sam Wiseman, Kevin Gimpel

Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging (2022.findings-acl)

Challenge:	Empirically, the model with the best performing syntactic and semantic representations gives rise to the most disentangled representations.
Approach:	They propose a generative model that uses latent variables to learn a sentence that uses both latent and latent representations.
Outcome:	The proposed model achieves better disentanglement between semantic and syntactic representations by training with multiple losses, including losses that exploit aligned paraphrastic sentences and word-order information.

Copied to clipboard

Houquan Zhou, Yang Li, Zhenghua Li, Min Zhang

Test-Time Code-Switching for Cross-lingual Aspect Sentiment Triplet Extraction (2025.naacl-long)

Challenge:	Large-scale pre-trained language models (PLMs) have made extraordinary progress in most NLP tasks, but they fail to achieve state-of-the-art (SOTA) performance.
Approach:	They propose a Guassian HMM variant for unsupervised POS tagging that incorporates contexualized word representations into the decoder.
Outcome:	The proposed model outperforms state-of-the-art models on Penn Treebank and multilingual Universal Dependencies treebank v2.0.

Copied to clipboard

Dongming Sheng, Kexin Han, Hao Li, Yan Zhang, Yucheng Huang, Jun Lang, Wenqiang Liu

Multilinguality

Challenge:	Aspect Sentiment Triplet Extraction (ASTE) is a thriving research area . current code-switching methods suffer from term boundary detection issues and out-of-dictionary problems.
Approach:	They propose a test-time code-switching framework which bridges the gap between bilingual training and monolingual test- time prediction.
Outcome:	The proposed framework achieves an average improvement of 3.7% on four cross-lingual datasets.

Generative Emotion Cause Triplet Extraction in Conversations with Commonsense Knowledge (2023.findings-emnlp)

Copied to clipboard

Fanfan Wang, Jianfei Yu, Rui Xia

Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding (2023.findings-acl)

Challenge:	Existing studies on ECTEC focus on Causal Emotion Entailment and Emotion-Cause Pair Extraction in Conversations.
Approach:	They propose to decompose the ECTEC task into multiple subtasks and solve them in a pipeline manner.
Outcome:	The proposed model outperforms competing systems on two benchmark datasets.

Copied to clipboard

Mirac Suzgun, Luke Melas-Kyriazi, Dan Jurafsky

Text Generation

Challenge:	Existing text decoding methods struggle to produce high-quality text . Greedy and beam search suffer from text degeneration and linguistic diversity issues .
Approach:	They propose a family of decoding methods based on minimum bayes risk minimization to address diversity-quality trade-offs in open-ended natural-language generation.
Outcome:	The proposed methods improve diversity-quality trade-offs on open-ended natural-language generation tasks.

Distilling the Knowledge of Large-scale Generative Models into Retrieval Models for Efficient Open-domain Conversation (2021.findings-emnlp)

Copied to clipboard

Beomsu Kim, Seokjun Seo, Seungju Han, Enkhbayar Erdenee, Buru Chang

Green & Sustainable NLP

Challenge:	generative models are less practical for building real-time conversation systems due to high latency and large memory footprint.
Approach:	They propose a method that preserves the efficiency of a retrieval model while leveraging the conversational ability of generative models.
Outcome:	The proposed method preserves the efficiency of a retrieval model while leveraging the conversational ability of generative models.

VCD: A Dataset for Visual Commonsense Discovery in Images (2025.findings-acl)

Copied to clipboard

Xiangqing Shen, Fanfan Wang, Siwei Wu, Rui Xia

An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extraction (N19-1)

Challenge:	Visual commonsense data sets lack visual grounded representations of commonsensense . existing knowledge bases lack visual-based knowledge tied to actual visual scenes .
Approach:	They present a large-scale visual commonsense dataset with over 100,000 images and 14 million object-commonsense pairs that integrates both Seen (directly observable) and Unseen (inferrable) commonsens.
Outcome:	The proposed model integrates Seen (directly observable) and Unseen (inferrable) commonsense across Property, Action, and Space aspects.

Copied to clipboard

Wang Chen, Hou Pong Chan, Piji Li, Lidong Bing, Irwin King

AISFG: Abundant Information Slot Filling Generator (2022.naacl-main)

Challenge:	Existing methods on keyphrase generation are purely extractive or generative . however, extractive methods cannot predict absent keyphrases which are not in the document.
Approach:	They propose a multi-task learning framework that jointly learns an extractive model and a generative model.
Outcome:	The proposed approach outperforms the state-of-the-art methods on five keyphrase generation tasks.

Copied to clipboard

Yang Yan, Junda Ye, Zhongbao Zhang, Liwen Wang

Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections (P19-1)

Challenge:	Existing approaches to zero/few-shot slot filling focus on slot descriptions and examples . AISFG model is based on domain-specific labels, which is not capable of transferring to new domains with little or no data.
Approach:	They propose a model with a query template that incorporates domain descriptions, slot descriptions, and examples with context.
Outcome:	Experimental results show that the proposed model outperforms state-of-the-art approaches in zero/few-shot slot filling task.

Copied to clipboard

Junxian He, Zhisong Zhang, Taylor Berg-Kirkpatrick, Graham Neubig

Syntactic Text Processing

Challenge:	Current systems for syntactic analysis tasks rely heavily on large scale annotated data.
Approach:	They propose to learn a generative model with a structured prior that uses labeled source and unlabeled target data jointly.
Outcome:	The proposed model improves on part-of-speech tagging and dependency parsing tasks on English as the only source corpus and on a wide range of target languages.

Exploring Key Point Analysis with Pairwise Generation and Graph Partitioning (2024.naacl-long)

Copied to clipboard

Xiao Li, Yong Jiang, Shen Huang, Pengjun Xie, Gong Cheng, Fei Huang

Structured Data in NLP

Challenge:	Existing methods for key point analysis rely on semantic similarity instead of measuring the existence of shared key points .
Approach:	They propose a key point analysis approach with pairwise generation and graph partitioning to summarize arguments into a concise set of key points.
Outcome:	The proposed model surpasses existing models on ArgKP and QAM datasets.

ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language (2021.findings-acl)

Copied to clipboard

Oyvind Tafjord, Bhavana Dalvi, Peter Clark

Linguistics & Cognitive NLP

Challenge:	Recent work shows that transformers can generate both implications of a theory and the natural language proofs that support them.
Approach:	They propose a generative model that generates both implications of a theory and natural language proofs that support them.
Outcome:	The proposed model generates both implications of a theory and the natural language proofs that support them.

Improving Compositional Generalization with Latent Structure and Data Augmentation (2022.naacl-main)

Copied to clipboard

Linlu Qiu, Peter Shaw, Panupong Pasupat, Pawel Nowak, Tal Linzen, Fei Sha, Kristina Toutanova

Generative Deduplication For Socia Media Data Selection (2024.findings-emnlp)

Challenge:	Generic unstructured neural networks struggle on out-of-distribution compositional generalization.
Approach:	They propose a method to recombinate examples from a model called Compositional Structure Learner and add them to a pre-trained sequence-to-sequence model.
Outcome:	The proposed model is even stronger than a T5-CSL ensemble on two real world compositional generalization tasks.

Copied to clipboard

Xianming Li, Jing Li

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint (2024.findings-acl)

Challenge:	Social media data exhibits severe redundancy due to its noisy nature, leading to increased training time and model bias in its processing.
Approach:	They propose a new framework for deduplication of social media data by removing semantically duplicate data from the model and add time-dimensional Gaussian noise to reduce training complexity.
Outcome:	The proposed framework can reduce training samples while improving performance over baselines.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, Ji-Rong Wen

SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain (2021.naacl-main)

Challenge:	Existing reinforcement learning methods do not provide fine-grained supervision for complex reasoning tasks.
Approach:	They propose a reinforcement learning method that incorporates a generative model as the reward model and a token-level supervision model for RL training.
Outcome:	Experiments on 8 tasks show the proposed method is effective .

Copied to clipboard

Ruochen Zhang, Carsten Eickhoff

Challenge:	Existing methods for state tracking are limited and state changes are less densely distributed over utterances.
Approach:	They propose to turn to simplified, fully observable systems that show some of these properties.
Outcome:	The proposed system shows that state changes occur infrequently while messages are "chatter" it allows for rich descriptions of state while avoiding the complexities of other settings.

Semi-supervised Autoencoding Projective Dependency Parsing (2020.coling-main)

Copied to clipboard

Xiao Zhang, Dan Goldwasser

Code-switched Language Models Using Dual RNNs and Same-Source Pretraining (D18-1)

Challenge:	Existing models for semi-supervised dependency parsing use labeled data, but they require large amounts of labeles.
Approach:	They propose two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing.
Outcome:	The proposed models outperform a semi-supervised model on WSJ and UD dependency parsing data sets.

Copied to clipboard

Saurabh Garg, Tanmay Parekh, Preethi Jyothi

Multilinguality

Challenge:	Using recurrent neural networks to build language models for code-switched text is an important problem with implications to downstream applications such as speech recognition and machine translation.
Approach:	They propose a novel recurrent neural network unit with dual components that focus on each language in the code-switched text separately and a generative model estimated using the training data.
Outcome:	The proposed techniques yield significant reductions in perplexity on Mandarin-English task and improve on baseline models.

Enhancing Reasoning Capabilities by Instruction Learning and Chain-of-Thoughts for Implicit Discourse Relation Recognition (2023.findings-emnlp)

Copied to clipboard

Yuxiang Lu, Yu Hong, Zhipang Wang, Guodong Zhou

Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space (2020.emnlp-main)

Challenge:	Existing models for implicit discourse relation recognition are based on generative models, but some studies suggest they do not perform as well as generic encoder-only models for NLU tasks.
Approach:	They propose a classification method that is solely based on generative models and utilize Chain-of-Thoughts to partition the inference process into a sequence of three successive stages.
Outcome:	The proposed model outperforms existing models on a natural language understanding task.

Copied to clipboard

Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, Jianfeng Gao

Leveraging Meta Information in Short Text Aggregation (P19-1)

Challenge:	Existing models for language understanding and understanding can be trained to provide contextualized representations of words based on text data.
Approach:	They propose a large-scale language VAE model Optimus that is pre-trained on large text corpus and fine-tuned for various language generation and understanding tasks.
Outcome:	The proposed model achieves new state-of-the-art on VAE language modeling benchmarks.

Copied to clipboard

He Zhao, Lan Du, Guanfeng Liu, Wray Buntine

Data Augmentation with Adversarial Training for Cross-Lingual NLI (2021.acl-long)

Challenge:	Existing topic models infer topics based on word co-occurrence information, which results in degraded performance and degrades performance.
Approach:	They propose a generative model that aggregates short texts into clusters by leveraging the associated meta information.
Outcome:	The proposed model can generate more interpretable topics and document clusters.

Copied to clipboard

Xin Dong, Yaxin Zhu, Zuohui Fu, Dongkuan Xu, Gerard de Melo

NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries? (2022.findings-emnlp)

Challenge:	Existing approaches to train cross-lingual models with labeled data are subpar, resulting in subpar results.
Approach:	They propose a data augmentation strategy that enriches data to reflect more diversity in a semantically faithful way and leverages adversarial training regimens to achieve greater robustness.
Outcome:	The proposed approach improves cross-lingual inference by leveraging the data to reflect more diversity in a semantically faithful way.

Copied to clipboard

Saadia Gabriel, Hamid Palangi, Yejin Choi

Text Classification

Challenge:	Existing work has explored adversarial example generation for natural language understanding tasks, but these examples are often unrealistic and diverge from the real-world data distributions.
Approach:	They propose a framework for adversarial example generation that is effective at fooling a given classifier and a generative model based on the key tokens from the first stage.
Outcome:	The proposed framework generalizes across domains and offers insights for future research on improving robustness of neural text classification models.

A Unified Generative Approach to Product Attribute-Value Identification (2023.findings-acl)

Copied to clipboard

Keiji Shinzato, Naoki Yoshinaga, Yandi Xia, Wei-Te Chen

Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEs (2022.naacl-main)

Challenge:	Product attribute value identification (PAVI) is a core task in the e-commerce industry.
Approach:	They propose a generative approach to product attribute-value identification (PAVI) they use product text to decode a set of attribute- value pairs as a target sequence from the given product text.
Outcome:	The proposed approach outperforms extraction- and classification-based methods on large-scale real-world datasets.

Copied to clipboard

Ghazi Felhi, Joseph Le Roux, Djamé Seddah

Private prediction for large-scale synthetic text generation (2024.findings-emnlp)

Challenge:	Existing models for text generation do not need syntactic information such as constituency parses or semantic information such a paraphrase pairs.
Approach:	They propose a generative model which exhibits disentangled latent representations of syntax and semantics by using Attention in its decoder.
Outcome:	The proposed model outperforms supervised models on syntax/semantics transfer and shows that it can read latent variables with keys and values.

Copied to clipboard

Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, Sergei Vassilvitskii

NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic Regularization (2024.findings-emnlp)

Challenge:	Existing approaches to generate differentially private text using large language models are classified into several categories.
Approach:	They propose a private prediction framework that generates differentially private synthetic text using large language models via private prediction.
Outcome:	The proposed approach generates high-quality synthetic data points at reasonable privacy levels while protecting the privacy of users who contributed to the dataset.

Copied to clipboard

Duy-Tung Pham, Thien Trang Nguyen Vu, Tung Nguyen, Linh Ngo, Duc Nguyen, Thien Nguyen

Auto-Encoding Variational Bayes for Inferring Topics and Visualization (2020.coling-main)

Challenge:	Recent advances in neural topic models have focused on two main directions: the integration of the inference network with a pre-trained language model and the modeling of the relationship between words and topics in the generative model.
Approach:	They propose a framework to maximize mutual information between topics and PLMs . Experimental results indicate that NeuroMax reduces inference time .
Outcome:	The proposed framework reduces inference time, generates more coherent topics and topic groups, and produces more representative document embeddings.

Copied to clipboard

Dang Pham, Tuan Le

Unsupervised Opinion Summarization as Copycat-Review Generation (2020.acl-main)

Challenge:	Visualization and topic modeling are widely used for text analysis, but there is a lack of semantic interpretation as they do not extract topics.
Approach:	They propose a fast Auto-Encoding Variational Bayes based inference method for jointly inferring topics and visualization using a single generative model.
Outcome:	The proposed method can handle model changes efficiently with little rederivation effort.

Copied to clipboard

Arthur Bražinskas, Mirella Lapata, Ivan Titov

Unified Thinker: A General Reasoning Core for Image Generation (2026.acl-long)

Challenge:	Recent work on opinion summarization has focused on extracting fragments from reviews, but we use novel sentences to generate abstractive summaries.
Approach:	They propose an abstractive summarizer which does not use summaries in training and is trained end-to-end on a large collection of reviews.
Outcome:	The proposed model produces fluent and coherent summaries reflecting consensus opinions on Amazon and Yelp reviews.

Copied to clipboard

Sashuai Zhou, Qiang Zhou, Jijin Hu, Hanqing Yang, Yue Cao, Junpeng Ma, Yinchao Ma, Jun Song, Tiezheng Ge, Cheng Yu, Bo Zheng, Zhou Zhao

monoQA: Multi-Task Learning of Reranking and Answer Extraction for Open-Retrieval Conversational Question Answering (2022.emnlp-main)

Challenge:	generative models struggle with logic-intensive instruction following, exposing a persistent reasoning–execution gap.
Approach:	They propose a task-agnostic reasoning architecture for general image generation . they propose pixel-level feedback to ground the Thinker's policy in pixel feedback .
Outcome:	The proposed system significantly improves image reasoning and generation quality.

Copied to clipboard

Sarawoot Kongyoung, Craig Macdonald, Iadh Ounis

VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling (2020.emnlp-main)

Challenge:	Existing approaches to the Conversational Question Answering task have used multi-task learning to solve the task.
Approach:	They propose to use multi-task learning to improve the ORConvQA task by sharing the reranker and reader’s learned structure in a generative model.
Outcome:	The proposed model outperforms baseline models on the OR-QuAC and OR-CoQA datasets and significantly outperformed existing strong baseline models.

Copied to clipboard

Machel Reid, Edison Marrese-Taylor, Yutaka Matsuo

Representation Learning

Challenge:	Existing approaches for definition modeling combine distributional and lexical semantics in an implicit rather than direct way.
Approach:	They propose a model that introduces a continuous latent variable to model the relationship between a phrase and its definition.
Outcome:	The proposed model achieves state-of-the-art performance on four challenging benchmarks and the first non-English corpus.

Generating Hashtags for Short-form Videos with Guided Signals (2023.acl-long)

Copied to clipboard

Tiezheng Yu, Hanchao Yu, Davis Liang, Yuning Mao, Shaoliang Nie, Po-Yao Huang, Madian Khabsa, Pascale Fung, Yi-Chia Wang

Improving Aspect Sentiment Quad Prediction via Template-Order Data Augmentation (2022.emnlp-main)

Challenge:	Short-form video hashtag recommendation (SVHR) is a classification or ranking problem that selects hashtags from a set of limited candidates.
Approach:	They propose a short-form video hashtag recommendation task that better represents how hashtags are created naturally by retrieving relevant hashtags from a large-scale hashtag pool as extra guidance signals.
Outcome:	The proposed model outperforms strong classification baselines on two short-form video datasets and the guidance signals boost the performance by 8.11 and 2.17 absolute ROUGE-1 scores on average.

Copied to clipboard

Mengting Hu, Yike Wu, Hang Gao, Yinhao Bai, Shiwan Zhao

Learning the Beauty in Songs: Neural Singing Voice Beautifier (2022.acl-long)

Challenge:	Recent work on aspect sentiment quad prediction (ASQP) uses a template to extract aspect quadruplets from review sentences.
Approach:	They propose to use a pre-trained language model to select proper orders from a template order perspective to improve aspect sentiment quad prediction.
Outcome:	The proposed method outperforms state-of-the-art methods significantly in low-resource settings.

Copied to clipboard

Jinglin Liu, Chengxi Li, Yi Ren, Zhiying Zhu, Zhou Zhao

Speech & Audio in NLP

Challenge:	Existing techniques for pitch correction are limited to intonation but ignore the overall aesthetic quality.
Approach:	They propose a novel time-warping approach for pitch correction to synchronize the amateur recording with the template pitch curve.
Outcome:	The proposed model improves intonation and vocal tone while keeping content and vocal timbre.

Learning from Negative Samples in Biomedical Generative Entity Linking (2025.findings-acl)

Copied to clipboard

Chanhwi Kim, Hyunjae Kim, Sihyeon Park, Jiwoo Lee, Mujeen Sung, Jaewoo Kang

Quick Back-Translation for Unsupervised Machine Translation (2023.findings-emnlp)

Challenge:	Generative models are usually trained only with positive samples and do not explicitly learn from hard negative samples, which are entities that look similar but have different meanings.
Approach:	They propose a framework that trains generative BioEL models using negative samples to learn from hard negative samples.
Outcome:	The proposed framework outperforms baseline models by up to an average top-1 accuracy of 1.4% on five benchmarks.

Copied to clipboard

Benjamin Brimacombe, Jiawei Zhou

Confusionset-guided Pointer Networks for Chinese Spelling Check (P19-1)

Challenge:	Unsupervised machine translation models are limited by the run-time of autoregressive inference during back-translation and lack of synthetic data efficiency.
Approach:	They propose a two-for-one improvement to Transformer back-translation: Quick Back-Translation (QBT). QBT re-purposes the encoder as a generative model, and uses encoder-generated sequences to train the decoder.
Outcome:	Experiments on various WMT benchmarks show that QBT dramatically outperforms standard back-translation only method in terms of training efficiency for comparable translation qualities.

Copied to clipboard

Dingmin Wang, Yi Tay, Li Zhong

Text Error Correction

Challenge:	Existing methods to detect and fix errors in Chinese are limited due to context.
Approach:	They propose a Confusionset-guided pointer network for Chinese Spell Check task . they propose to use off-the-shelf confusionset to guide character generation .
Outcome:	The proposed model outperforms all competitor models on three human-annotated datasets.

The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers (2023.acl-long)

Copied to clipboard

Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, Eyal Shnarch

FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning (2022.acl-long)

Challenge:	Existing approaches to apply language models to tasks that require intermediate representations are less informative.
Approach:	They propose a novel approach that utilizes the contrast between layers to improve text generation outputs.
Outcome:	The proposed approach mitigates degenerative behaviors of the model in open-ended generation, significantly improving the quality of generated texts.

Copied to clipboard

Jing Zhou, Yanan Zheng, Jie Tang, Li Jian, Zhilin Yang

Semantic and Sentiment Dual-Enhanced Generative Model for Script Event Prediction (2025.coling-main)

Challenge:	Existing methods for text data augmentation are limited to simple tasks and weak baselines.
Approach:	They propose a data augmentation method FlipDA that uses a generative model and a classifier to generate label-flipped data.
Outcome:	The proposed method improves many tasks while not negatively affecting the others.

Copied to clipboard

Feiyang Wu, Peixin Huang, Yanli Hu, Zhen Tan, Xiang Zhao

Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction (2024.acl-long)

Challenge:	Existing methods to model event associations struggle with semantic ambiguity and embedding bias.
Approach:	They propose a Semantic and Sentiment Dual-enhanced Generative Model to address these issues . it leverages two types of script event information to enhance the generative model .
Outcome:	The proposed model captures both global and local sentiments of events through its sentiment awareness mechanism.

Copied to clipboard

Yice Zhang, Jie Zeng, Weiming Hu, Ziyi Wang, Shiwei Chen, Ruifeng Xu

GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks (2023.findings-acl)

Challenge:	Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review.
Approach:	They propose a self-training framework with a pseudo-label scorer to assess the match between reviews and their pseudo-labels and train a generative model on it.
Outcome:	The proposed framework can predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review, and it can significantly improve self-training.

Copied to clipboard

Xuming Hu, Aiwei Liu, Zeqi Tan, Xin Zhang, Chenwei Zhang, Irwin King, Philip S. Yu

Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval (2023.acl-long)

Challenge:	Existing work adopts data augmentation techniques to generate pseudo-annotated sentences . existing methods neither preserve semantic consistency of original sentences nor preserve syntax structure of sentences when expressing relations using seq2seq models, resulting in less diverse augmentations.
Approach:	They propose a dedicated augmentation technique for relational texts, named GDA, which uses two complementary modules to preserve both semantic consistency and syntax structures.
Outcome:	The proposed technique can bring 2.0% F1 improvements in three datasets under low-resource setting.

Copied to clipboard

John Wieting, Jonathan Clark, William Cohen, Graham Neubig, Taylor Berg-Kirkpatrick

Bidirectional Generative Framework for Cross-domain Aspect-based Sentiment Analysis (2023.acl-long)

Challenge:	Contrastive learning is the dominant paradigm for learning text representations from parallel text, but finding negative examples can be expensive in terms of compute or manual effort.
Approach:	They propose a generative model for learning multilingual text embeddings which encourages source separation in multilingual contexts by an approximation.
Outcome:	The proposed model outperforms both a strong contrastive and generative baseline on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.

Copied to clipboard

Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, Lidong Bing

On the Strength of Sequence Labeling and Generative Models for Aspect Sentiment Triplet Extraction (2023.findings-acl)

Challenge:	Aspect-based sentiment analysis (ABSA) is a task of analyzing people's sentiments at the aspect level.
Approach:	They propose a unified bidirectional generative framework to tackle cross-domain ABSA tasks . the framework trains a model in both text-to-label and label-totext directions .
Outcome:	The proposed framework trains a model in both label-to-label and label- to-text directions to learn domain-agnostic features.

Copied to clipboard

Shen Zhou, Tieyun Qian

Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022.emnlp-main)

Challenge:	Existing generative models ignore the mutual informative clues between aspect and opinion terms and may generate false paired triplets.
Approach:	They propose a generative model that encodes aspect and opinion into two bidirectional templates and introduces a marker-oriented sequence labeling module to improve models’ ability of tackling complex structures.
Outcome:	The proposed model captures the boundary information of aspect/opinion spans and provides hints to decode multiple triplets with the shared marker.

Copied to clipboard

Yingxiu Zhao, Yinhe Zheng, Zhiliang Tian, Chang Gao, Jian Sun, Nevin L. Zhang

UniCOQE: Unified Comparative Opinion Quintuple Extraction As A Set (2023.findings-acl)

Challenge:	Existing generative replay methods use only a single task-specific token to control their models.
Approach:	They propose a method to capture task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation.
Outcome:	The proposed method outperforms baselines on natural language understanding tasks of advanced task-oriented dialogue (ToD) systems.

Copied to clipboard

Zinong Yang, Feng Xu, Jianfei Yu, Rui Xia

Generative Adversarial Training with Perturbed Token Detection for Model Robustness (2023.emnlp-main)

Challenge:	Existing methods decompose the COQE task into multiple subtasks and solve them in a pipeline manner, but ignore the intrinsic connection between subtask and the error propagation among stages.
Approach:	They propose a unified generative model that solves COQE in one shot by concatenating all the comparative tuples into a target output sequence.
Outcome:	The proposed model significantly outperforms the SOTA method on multiple benchmarks and ablation experiments.

Copied to clipboard

Jiahao Zhao, Wenji Mao

AudioVSR: Enhancing Video Speech Recognition with Audio Data (2024.emnlp-main)

Challenge:	Existing adversarial training methods use discrete tokens to deceive models . current approaches use embeddings, whereas actual text-based training uses discrete text tokens.
Approach:	They propose a framework that integrates gradient-based learning, adversarial example generation and perturbed token detection to enhance adversariarial robustness.
Outcome:	The proposed framework surpasses the state-of-the-art results of ChatGPT by 10% in average accuracy.

Copied to clipboard

Xiaoda Yang, Xize Cheng, Jiaqi Duan, Hongshun Qiu, Minjie Hong, Minghui Fang, Shengpeng Ji, Jialong Zuo, Zhiqing Hong, Zhimeng Zhang, Tao Jin

RRInf: Efficient Influence Function Estimation via Ridge Regression for Large Language Models and Text-to-Image Diffusion Models (2025.emnlp-main)

Challenge:	Recent work has shown poor performance with non-Indo-European languages . previous work primarily utilizes video information to build VSR models .
Approach:	They propose a generative model for data inflation that integrates synthetic data with authentic visual data to enhance the VSR model.
Outcome:	The proposed model improves on the audio-visual alignment problem in audio-video tasks.

Copied to clipboard

Zhuozhuo Tu, Cheng Chen, Yuxuan Du

Is the Answer in the Text? Challenging ChatGPT with Evidence Retrieval from Instructive Text (2023.findings-emnlp)

Challenge:	Influence function estimates the impact of training data on model predictions . high computational cost has hindered their applicability in large-scale applications.
Approach:	They propose a method to quantify the impact of training data on model predictions . they use a ridge regression problem to transform the estimation into a problem .
Outcome:	The proposed method outperforms existing methods on noisy data detection and influential data identification tasks.

Copied to clipboard

Sophie Henning, Talita Anthonio, Wei Zhou, Heike Adel, Mohsen Mesgar, Annemarie Friedrich

GenKIE: Robust Generative Multimodal Document Key Information Extraction (2023.findings-emnlp)

Challenge:	Generative language models have recently shown remarkable success in generating answers to questions in a given textual context, but they suffer from hallucination, wrongly cite evidence, and spread misleading information.
Approach:	They propose a benchmark to evaluate an annotated WikiHow article and use it to retrieve answers to questions from trustworthy texts.
Outcome:	The proposed model can retrieve answers to lexically varied and open-ended questions from trustworthy instructive texts.

Copied to clipboard

Panfeng Cao, Ye Wang, Qiang Zhang, Zaiqiao Meng

Robustness in NLP

Challenge:	Key information extraction (KIE) is a key application for information retrieval and text mining.
Approach:	They propose a novel generative end-to-end model, named GenKIE, to address the KIE task.
Outcome:	The proposed model generalizes over different types of documents and achieves state-of-the-art results.

Prefix-Tuning Based Unsupervised Text Style Transfer (2023.findings-emnlp)

Copied to clipboard

Huiyu Mai, Wenhao Jiang, Zhi-Hong Deng

Low-Resource NLP

Challenge:	Unsupervised text style transfer is an important task in computer vision and natural language processing.
Approach:	They propose a method that uses pre-trained large language models to train a generative model that can alter the style of the input sentence without using any parallel data.
Outcome:	The proposed method outperforms the state-of-the-art methods on well-known datasets.

T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics (2023.findings-emnlp)

Copied to clipboard

Yiwei Qin, Weizhe Yuan, Graham Neubig, Pengfei Liu

ALGEN: Few-shot Inversion Attacks on Textual Embeddings via Cross-Model Alignment and Generation (2025.acl-long)

Challenge:	Existing methods for evaluating text quality are discriminative and generative . current methods use manual annotation of human judgements to train them .
Approach:	They propose a framework that combines the best of both worlds by using supervised and unsupervised signals from whatever data we have available.
Outcome:	The proposed method outperforms existing metrics on 5 datasets, 19 languages and 280 systems.

Copied to clipboard

Yiyi Chen, Qiongkai Xu, Johannes Bjerva

Scale-VAE: Preventing Posterior Collapse in Variational Autoencoder (2024.lrec-main)

Challenge:	Recent studies have proven that private textual data is vulnerable to inversion attacks . authors present a method to reduce the cost of inversion and improve performance .
Approach:	They propose a method that aligns victim embeddings to attack space and reconstructs text . they find that none of the defense mechanisms are effective against inversion attacks .
Outcome:	The proposed method lowers the cost of inversion and improves performance across languages and domains.

Copied to clipboard

Tianbao Song, Jingbo Sun, Xin Liu, Weiming Peng

Beyond the Safety Tax: Mitigating Unsafe Text-to-Image Generation via External Safety Rectification (2026.findings-acl)

Challenge:	Variational autoencoder (VAE) is a widely used generative model . but when employing strong autoregressive generation networks, VAE tends to converge to a degenerate local optimum known as posterior collapse.
Approach:	They propose a model called Scale-VAE to solve a posterior collapse problem . they use a factor to keep the posterior dimension discriminative across data instances .
Outcome:	The proposed model outperforms state-of-the-art models in density estimation and representation learning.

Copied to clipboard

Xiangtao Meng, Yingkai Dong, Ning Yu, Li Wang, Zheng Li, Shanqing Guo

ToNER: Type-oriented Named Entity Recognition with Generative Language Model (2024.lrec-main)

Challenge:	Existing safety defenses typically intervene internally within the generative model, but suffer from severe concept entanglement, leading to degradation of benign generation quality.
Approach:	They propose a structurally isolated safety module that performs external, interpretable rectification without modifying the base model.
Outcome:	The proposed module performs external, interpretable rectification without modifying the base model.

Copied to clipboard

Guochao Jiang, Ziqin Luo, Yuchen Shi, Dixuan Wang, Jiaqing Liang, Deqing Yang